Controller for photographing apparatus and photographing system

ABSTRACT

A picture photographed by a camera portion is sent to a video capturing portion of a computer. The picture is displayed in an operation area of a monitor. A panorama picture of which pictures in part or all moving range of a pan tiler are combined is displayed in a panorama operation area. A pan tilter portion sends positional information of pan and tilt to the computer through a mode controller. With a mouse, the operation area and the panorama operation area are operated so as to select an object. The computer obtains data for driving the pan tilter. Thus, the selected object is displayed at the center of the operation area.

This is a continuation application of Ser. No. 13/252,243, filed Oct. 4,2011, which is a continuation of application Ser. No. 12/505,814, filedJul. 20, 2009, now U.S. Pat. No. 8,045,837, issued Oct. 25, 2011, whichis a continuation of application Ser. No. 12/269,095, filed Nov. 12,2008, now U.S. Pat. No. 7,720,359, issued May 18, 2010, which is acontinuation of application Ser. No. 10/766,405, filed Jan. 27, 2004,now abandoned, which is a continuation of application Ser. No.09/059,774, filed Apr. 14, 1998, now U.S. Pat. No. 6,720,987, issuedApr. 13, 2004.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a controller for a photographingapparatus and a photographing system with a high operationalcharacteristic and a high visibility suitable for a photographingoperation of the apparatus that is disposed at a remote place and thatis used for a monitoring operation, an observing operation, a guidingoperation, a presenting operation, and so forth.

2. Description of the Related Art

As shown in FIG. 23, when the user controls a photographing apparatusdisposed at a remote place, he or she operates a pan tilter in eightdirections (up, down, left, right, upper right, lower right, upper left,and lower left directions) with eight-direction keys, a zoomingcontroller, and a wide-angle controller so as to photograph a desiredobject while observing a photographed picture 6A on a monitor 2. In thestructure shown in FIG. 23, the user moves a cursor 7 to one of thedirection keys 10 with a mouse 8. Alternatively, after the user hascontrolled a photographing apparatus disposed at a remote place in theabove-described method and registered pan tilter information and zoominformation of positions of pictures to be photographed, he or shedrives the photographing apparatus at absolute positions correspondingto the registered positions so as to select pictures.

In the conventional controller, a picture that is displayed on themonitor is limited in the range of which the photographing apparatus ismoved by the pan tilter. Thus, when the user photographs a desiredobject, he or she should operate the pan tilter in the full rangethereof. Consequently, the user should have skill in operating the pantilter.

When the user changes the photographing direction with the conventionaldirection keys, even if he or she stops pressing the direction keys,since the pan tilter does not immediately stops and thereby he or shemay not catch a desired object. When the direction varying speed of thephotographing apparatus with the pan tilter is low, although such aproblem may be solved, since the response characteristic deteriorates, ahigh operational characteristic cannot be obtained.

When the user wants to place a desired object at the center of the angleof view of the photographing apparatus, since he or she controls thephotographing direction while observing a picture on the monitor, he orshe should determine the photographing direction on trial and errorbasis. Thus, the user may spend a long time for controlling thephotographing apparatus. Moreover, to properly operate the photographingapparatus, the user should have skill.

When picture and control information is exchanged with a photographingapparatus disposed at a remote place through a low-capacity network, thecontrol information may be lost and/or picture information may bedelayed due to an irregularity of their arrival intervals. If the pantilter or the zooming controller is operated for picture and controlinformation that has been delayed or lost, even if the user causes thepan tilter and the zooming controller to place the object at the desiredposition, the pan tilter and the zooming controller do not properlyoperate. Thus, the object is placed at an improper position due to thedelay. In addition, depending on the line condition, the arrivalintervals of picture information vary. Thus, the user should control thepan tilter and the zooming controller based on a prediction.Consequently, the user cannot properly control the pan tilter and thezooming controller.

OBJECTS AND SUMMARY OF THE INVENTION

Therefore, an object of the present invention is to provide a controllerfor a photographing apparatus for allowing the user to designate adesired position or a desired area on a panorama picture displayed as apart or all the moving range of a pan tilter so that the user can easilyobtain a desired picture with the photographing apparatus.

Another object of the present invention is to provide a controller for aphotographing apparatus and a photographing system with a highvisibility and a high operational characteristic that allow the user todesignate a desired position or a desired area on a screen and select anobject with the designated position or area and the photographingapparatus to place the selected object at the center of the screen.

A first aspect of the present invention is a controller for aphotographing apparatus having a photographing portion with drivingmeans that allows the photographing direction of photographing means tobe varied, comprising a displaying means for displaying a panoramapicture generated with a picture photographed by the photographingmeans, and a controlling means for referencing the panorama picture andvarying the photographing direction of the photographing means.

A second aspect of the present invention is a controller for aphotographing apparatus having a photographing portion with drivingmeans that allows the photographing direction of photographing means tobe varied, the controller comprising an operation area in which apanorama picture generated with a picture photographed by thephotographing means is displayed, and a picture selecting means forallowing the user to designate a desired point in the operation area,selecting an object photographed by the photographing meanscorresponding to the designated point, and moving the selected object todesired positional coordinates of the driving means.

A third aspect of the present invention is a controller for aphotographing apparatus having a photographing portion with drivingmeans that allows the photographing direction of photographing means tobe varied, the controller comprising an operation area in which apanorama picture generated with a picture photographed by thephotographing means is displayed, and a picture selecting means forallowing the user to designate a desired area in the operation area,selecting an object photographed by the photographing meanscorresponding to the designated area, and moving an object at theposition corresponding to a desired point generated with the desiredarea to desired positional coordinates of the driving means.

A fourth aspect of the present invention is a photographing systemhaving a photographing portion with driving means that allows thephotographing direction of photographing means to be varied and acontroller for a photographing apparatus, the controller controlling thephotographing portion, wherein the controller comprises an operationarea in which a panorama picture generated with a picture photographedby the photographing means is displayed, and a picture selecting meansfor selecting an object photographed by the photographing means in theoperation area and moving the selected object to desired positionalcoordinates of the driving means.

A picture photographed by a pan tilter camera that is disposed at aremote place and that can be moved in various directions is sent to acomputer. The picture is displayed as a panorama picture in a displayarea of a monitor. The direction of a picture selecting meanscorresponding to the direction of an object to be placed at the centerof the angle of view of the photographing apparatus in the panoramapicture is designated by a pointing device connected to the computer.Since the pan tilter is controlled with reference to the panoramapicture, a desired picture can be photographed by the photographingapparatus.

In addition, the environment of the place at which the pan tilter camerais disposed is displayed as a panorama picture in the panorama operationarea of the monitor of the computer. A desired point to be placed at thecenter of the angle of view of the photographing apparatus in a pictureof the panorama operation area or a desired point generated with adesired area is designated by the pointing device connected to thecomputer. Thus, in the method of which the result is input, a selectedobject can be easily placed at the center of the screen. In addition,since a desired point in the operation area on the screen or a desiredpoint generated with a desired area is designated with the pointingdevice, the user can easily know the driving direction of the pan tiltercamera. In addition to the panorama operation area, another operationarea for a picture may be displayed.

These and other objects, features and advantages of the presentinvention will become more apparent in light of the following detaileddescription of a best mode embodiment thereof, as illustrated in theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external view for explaining a system according to anembodiment of the present invention;

FIG. 2 is a schematic diagram for explaining a screen of a monitoraccording to the embodiment of the present invention;

FIG. 3 is a block diagram showing the structure of the system accordingto the embodiment of the present invention;

FIGS. 4A to 4F are schematic diagrams for explaining a method forgenerating a panorama picture according to the embodiment of the presentinvention;

FIGS. 5A to 5D are schematic diagrams for explaining a method forgenerating a panorama picture according to the embodiment of the presentinvention;

FIGS. 6A to 6C are schematic diagrams for explaining a method forgenerating a panorama picture according to the embodiment of the presentinvention;

FIGS. 7A and 7B are schematic diagrams for explaining a method forgenerating angular information of a pan tilter camera with positionalcoordinates in a panorama operation area according to the embodiment ofthe present invention;

FIGS. 8A and 8B are schematic diagrams for explaining a plane-sphericalsurface converting method according to the embodiment of the presentinvention;

FIGS. 9A and 9B are schematic diagrams for explaining a coordinateconverting method in the operation area according to the embodiment ofthe present invention;

FIGS. 10A to 10C are schematic diagrams for explaining a coordinateconverting method in the panorama operation area according to theembodiment of the present invention;

FIGS. 11A and 11B are schematic diagrams for explaining positionalinformation and angular information of a pan tilter camera according tothe embodiment of the present invention;

FIGS. 12A and 12B are schematic diagrams for explaining angularcoordinates of the pan tilter camera and positional coordinates in thepanorama operation area according to the embodiment of the presentinvention;

FIGS. 13A to 13D are schematic diagrams for explaining the angle of viewof the pan tilter camera and a frame in the panorama operation areaaccording to the embodiment of the present invention;

FIG. 14 is a graph for explaining a conversion method of zoom data andmagnification data according to the embodiment of the present invention;

FIG. 15 is a flow chart showing an example of the overall processaccording to the embodiment of the present invention;

FIGS. 16A and 16B are flow charts showing an example of the process of atimer event according to the embodiment of the present invention;

FIG. 17 is a flow chart showing an example of the process of a mousemoving event according to the embodiment of the present invention;

FIG. 18 is a flow chart showing an example of the process of a mousebutton down event according to the embodiment of the present invention;

FIG. 19 is a flow chart showing another example of the process of amouse button down event according to the embodiment of the presentinvention;

FIG. 20 is a flow chart showing an example of the process of a mouseup/down event according to the embodiment of the present invention;

FIG. 21 is a schematic diagram showing the structure of a systemaccording to a second embodiment of the present invention;

FIG. 22 is a block diagram showing the structure of the system accordingto the second embodiment of the present invention; and

FIG. 23 is a schematic diagram for explaining a controller for aphotographing apparatus.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Next, with reference to the accompanying drawings, embodiments of thepresent invention will be described. FIG. 1 shows an outline of thestructure of a system according to a first embodiment of the presentinvention. A monitor 2 and a mouse 8 are connected to a computer 1. Thecomputer 1 controls the driving operation of a pan tiler camera 3disposed at a remote place. In other words, a controller for thephotographing apparatus is composed of the computer 1.

The pan tilter camera 3 is integrally composed of a pan tilter portionand a camera portion. In FIG. 1, the pan tilter camera 3 is disposed ona real scene as denoted by 4. A screen of a picture photographed by thepan tilter camera 3 is denoted by 5. This screen is hereinafter referredto as a photographed screen. The photographed screen 5 is an actuallyphotographed screen. When a zoom lens of the pan tilter camera 3 isplaced on the telephotograph side, the angle of view decreases. Incontrast, when the zoom lens of the pan tilter camera 3 is placed on thewide-angle side, the angle of view increases.

A picture on a photographed screen 5 captured by a pan tilter camera 3is sent to a computer 1 through a video cable or the like. The picturedata sent to the computer 1 is decoded and displayed on a monitor 2. Themonitor 2 displays the photographed screen 5 in an operation area 6A onthe monitor 2. A panorama picture including the picture photographed bythe pan tilter camera 3 is displayed in a panorama operation area 6B. Anarrow-shaped cursor 7 is displayed at the position of a mouse pointer ofa mouse 8 in the operation area 6A or the panorama display area 6B. Theuser designates a desired point or a desired area in the operation area6A or the panorama operation area 6B with the mouse 8 so as to operatethe pan tilter camera 3. In the panorama operation area 6B, a frame 6Cthat represents the current position and the angle of view of the pantilter and a pan tilter limiter 6D are superimposed to the panoramapicture. The pan tilter limiter 6D represents the moving range of thepan tilter camera. In addition, when necessary, a panorama generationbutton 6E is displayed on the monitor 2.

As shown in FIG. 2, the operation area 6A and the panorama operationarea 6B are displayed on the monitor 2. With the mouse 8, the user canmove the cursor 7 and designate a desired point or a desired pointgenerated with a desired area in the operation area 6A or the panoramaoperation area 6B. The user operates the pan tiler so that an objectcorresponding to the designated point is placed at the center of theoperation area 6A. In other words, when the user inputs a result to bedisplayed, an object selected corresponding to the input data isdisplayed at the center of the operation area 6A.

FIG. 3 is a block diagram showing the overall system according to theembodiment of the present invention. The system shown in FIG. 3comprises a camera portion 11, a pan tilter portion 12, a TV monitor 13,a computer 1, a pointing device 14 (such as a mouse 8), and a monitor 2.The pan tilter camera 3 comprises a camera portion 11 and a pan tilterportion 12. For example, the came a portion 11 is disposed on the pantilter portion 12. The camera portion 11 comprises a lens block portion15, a zoom lens 16, a zoom portion 17, a zoom lens motor 18, a solidstate image pickup device 19, a signal separating/automatic gainadjusting circuit (SH/AGC) 20, an A/D converter 21, and a signalprocessing circuit 22. The camera portion 11 represents a video camera.

The pan tilter portion 12 comprises a mode controller 23, a cameracontroller 24, a pan tilter controller 25, a pan motor 26, a tilt motor27, and a pan tilter 28. The computer 1 comprises a controlling portion31, a video capture portion 29, and a storing portion 30. The videocapture portion 29 is composed of a video capture board.

Rays emitted from an object are focused to the solid state image pickupdevice 19 through a lens set and a diaphragm of the lens block portion15. An example of the solid state image pickup device 19 is a CCD(Charge Coupled Device). The focused rays (field picture) are convertedinto a picture signal and then sent to the signal separating/automaticgain adjusting circuit 20. The signal separating/automatic gainadjusting circuit 20 samples/holds the picture signal and controls thegain of the picture signal with a control signal of an auto iris (AE).The resultant picture signal is sent to the signal processing circuit 22through the A/D converter 21. The signal processing circuit 22 convertsthe received picture signal into a brightness signal (Y), a color signal(C), and a video signal and sends these signals as picture signals tothe TV monitor 13 and the video capture portion 29 of the computer 1.

The lens block portion 15 of the camera portion 11 drives the zoom lens16 and thereby varies the angle of view of an object to be photographed.The lens block portion 15 causes the zoom lens motor 18 that is forexample a stepping motor to rotate, thereby driving the zoom lens 16corresponding to a drive command received from the camera controller 24of the pan tilter portion 12. The camera controller 24 performs a lenscontrolling operation (for example, focusing operation and zoomingoperation), an exposure controlling operation (for example, diaphragmcontrolling operation, gain controlling operation, and speed controllingoperation of electronic shutter), white balance controlling operation, apicture quality controlling operation, and so forth of the cameraportion 11. In addition, the camera controller 24 interfaces with themode controller 23. As interface controlling operations with respect tothe zoom lens 16, the camera controller 24 sends a control signal to themotor driver corresponding to a drive command of the zoom lens 16received from the mode controller 23 so that the zoom lens 16 is placedat the position designated by the command. In addition, the cameracontrollers 24 always sends positional information of the zoom lens 16to the mode controller 23.

The camera portion 11 is disposed on the pan tilter portion 12 that hasa degree of freedom that are rotating directions of two axes of pan andtilt. The pan tilter portion 12 causes the pan motor 26 and the tiltmotor 27 to rotate corresponding to a drive command received from thepan tilter controller 25, thereby driving a pan head and a tilt head ofthe pan tilter 28. The motors 26 and 27 are composed of for examplestepping motors. The pan tilter controller 25 sends a control signal tothe motor drivers so that the pan head and the tilt head are driven topositions corresponding to a pan drive command and a tilt drive commandreceived from mode controller 23. In addition, the pan tilter controller25 always sends positional information of the pan head and the tilt headto the mode controller 23.

The mode controller 23 controls the overall system corresponding to theinternal states of the camera portion 11 and the pan tilter portion 12and the interface information received from the outside of the pantilter camera 3 as will be described later. The mode controller 23 isconnected with for example the computer 1 and RS-232C interface. Themode controller 23 sends drive commands received from the computer 1 tothe pan tilter controller 25 and the camera controller 24 so as to drivethe pan tilter 28 and the zoom lens 16 of the lens block portion 15. Inaddition, the mode controller 23 sends current positional informationreceived from the pan tilter controller 25 and the camera controller 24to the computer 1.

According to the embodiment, the computer 1 is used to select a picturephotographed by the pan tilter camera 3. The computer 1 processesgraphics in the operation area 6A and the panorama operation area 6Bdisplayed on the monitor 2 and information of a designated position anda clicking operation of the pointing device 14 (mouse 8) and sends theresultant data to the mode controller 23. To display a picturephotographed by a camera portion 11 on the monitor 2, a video capturingportion 29 is used. The video capturing portion 29 allows a video signalreceived from the camera portion 11 to be displayed on the monitor 2with a desired picture quality. In addition, the video capturing portion29 allows a picture to be captured in a particular picture format (forexample, bit map format, still picture JPEG format, moving picture JPEGformat, or the like) with a particular picture quality and to be storedin the storing portion 30 (for example, a hard disk) of the computer 1.

Next, with reference to FIG. 4, an example of a method for generating apanorama picture displayed in the panorama operation area 6B will bedescribed. It should be noted that according to the present invention, apanorama picture may be generated by another method. When the panoramageneration button 6E is pressed, a panorama picture is generated.

Now, it is assumed that the environment in the place where the pantilter camera 3 is disposed is a spherical surface. The sphericalsurface is referred to as virtual spherical surface. In FIGS. 4A to 4F,two adjacent pictures on the virtual spherical surface are combined toone panorama picture. To generate a panorama picture, as shown in FIG.4A, the pan tilter camera 3 disposed at the center of the spherephotographs two adjacent pictures on the virtual spherical surface. Thepan tilter camera 3 photographs a plane perpendicular to the opticalaxis of the lens thereof. FIG. 4D shows a situation of which twoadjacent pictures on the virtual spherical surface are photographed bythe pan tilter camera 3 and the two pictures are mapped to the planeperpendicular to the optical axis. When two adjacent pictures are simplycombined, they overlap and distort at the overlapped portion.

To prevent two adjacent pictures from overlapping and distorting, theyare mapped to the virtual spherical surface as shown in FIG. 4B. FIG. 4Eshows a situation of which) two photographed pictures that are planesperpendicular to the optical axis are mapped to the virtual sphericalsurface. In such a manner, planes perpendicular to the optical axis(namely, photographed pictures) are mapped to the virtual sphericalsurface. The mapped pictures are combined in such a manner that anoverlapped portion and an unnecessary portion are removed. The picturemapped on the virtual spherical surface is normalized with longitude andlatitude. Thus, a panorama picture as shown in FIGS. 4C and 4D isgenerated.

Next, a method for generating a panorama picture will be described. Inthis method, as shown in FIGS. 5A to 5D, one panorama picture isgenerated by combining 10 pictures. First, the pan tilter camera 3 (notshown) disposed at the center of the sphere photographs 10 pictures. Atthis point, as shown in FIG. 5A, by matching the optical axis of thelens of the pan tilter camera 3 to positions denoted by circles, the pantilter camera 3 can obtain pictures 1 to 10. As shown in FIG. 5B, thepictures photographed by the pan tilter camera 3 are pictures on theplane perpendicular to the optical axis of the lens. The obtainedpictures are mapped to the virtual spherical surface. Thereafter, asshown in FIG. 5C, the pictures are normalized with latitude andlongitude. The pictures are obtained in such a manner that they aresmoothly combined without a break. Thereafter, an overlapped portion andunnecessary portion are removed. Thus, a panorama picture of which 10picture are smoothly combined is generated.

Next, with reference to FIG. 6, another method for generating a panoramapicture will be described. In this method, pixels obtained by the pantilter camera 3 are designated to pixels of a panorama picturenormalized with latitude and longitude (namely, coordinates (s, t)). Asin the method shown in FIGS. 5A to 5D, when pixels of picturesphotographed by the pan tilter camera 3 are designated to pixels of apanorama picture, part of pixels of the panorama picture may not bedesignated. All pixels of pictures photographed by the pan tilter camera3 should be designated to pixels of the panorama picture. The panoramapicture is composed of pixels calculated for individual coordinatepoints in the following process. Angular coordinates (α, β) (see FIG.6B) on the virtual spherical surface corresponding to coordinates (s, t)(see FIG. 6) of a panorama picture are calculated corresponding to Eq.(1)(α,β)=(a(s),b(t))  (1)(Eq. (1) will be described later with reference to FIGS. 7A and 7B.)

As shown in FIG. 6C, coordinate data (ξ, η) of the obtained picture iscalculated with the coordinates (s, t), the angular coordinates (θ, φ)of a pan tilter 28, and photographing magnification assuming that thewide edge of the photographing apparatus is defined as one magnificationcorresponding to Eq. (2)(ξ,η)=(f(α,β,θ,φ,γ),g(α,β,θ,φ,γ))  (2)(Eq. (2) will be described later with reference to FIGS. 8A and 8B.)

Corresponding to the above-described equations, pixels of the panoramapicture are correlated with obtained pictures so as to generate acombined picture (namely, the panorama picture).

Next, with reference to FIGS. 7A and 7B, a method for convertingcoordinates (s, t) of a panorama picture into angular coordinates (α, β)on the virtual spherical surface will be described. In FIG. 7A, PragMinrepresents angular data at the left edge assuming that the home positionof the pan tilter 28 (for example, the center in the moving range of thepan tilter 28) is 0 (rag). PragMax represents angular data at the rightedge assuming that the home position of the pan tilter 28 is 0 (reg).Ny₂ represents a horizontal coordinate of the panorama operation area6B. −Ny₂/2 represents coordinate data at the right edge of the panoramaoperation area 6B.

To obtain the pan angle α with the coordinate data s, since thefollowing relation is satisfied(PragMax−α):(PragMax−PragMin)=(Ny ₂/2−s):Ny ₂the pan angle α is expressed as follows.α=PragMax−(PragMax−PragMin)×(Ny ₂/2−s)/Ny ₂

In FIG. 7B, TragMin represents angular data at the upper edge assumingthat the home position of the pan tilter 28 is 0 (rag). TragMaxrepresents angular data at the lower edge assuming that the homeposition of the pan tilter 28 is 0 (rag). Nz₂ represents a verticalcoordinate of the panorama operation area 6B. −Nz₂/2 representscoordinate data at the upper edge of the panorama operation area 6B.Nz₂/2 represents coordinate data at the lower edge of the panoramaoperation area 6B.

To obtain the tilt angle β with the coordinate data t, since thefollowing relation is satisfied,(TragMax−β):(TragMax−TragMin)=(Nz ₂/2−t):Nz ₂the tilt angle β is expressed as follows.β=TragMax−(TragMax−TragMin)×(Nz ₂/2−t)/Nz ₂

Next, with reference to FIGS. 8A and 8B, the method for converting aplane into a spherical surface will be described. As shown in FIG. 8A,the spatial coordinates of a point (ξ, η) of a photographed pictureorienting the home position (the origin of latitude and longitude) areexpressed as follows.

$\begin{matrix}{P = {\theta_{x} + {k_{1}{\xi\theta}_{\xi}} + {k_{2}{\eta\theta}_{n}}}} \\{= {\begin{bmatrix}1 \\0 \\0\end{bmatrix} + {k_{1}{\xi\begin{bmatrix}0 \\1 \\0\end{bmatrix}}} + {k_{2}{\eta\begin{bmatrix}0 \\0 \\1\end{bmatrix}}}}} \\{= \begin{bmatrix}1 \\{{- k_{1}}\xi} \\{k_{2}\eta}\end{bmatrix}}\end{matrix}$

At this point, the following relations are satisfied.k ₁=tan(λ/2λ)/(Ny/2)k ₂=tan(μ/2r)/(Nz/2)where (Ny, Nz) represent the drive ranges (y direction and z direction)of the mouse pointer of the pointing device 14 (mouse 8); (λ, μ)represents the horizontal angle of view and vertical angle of view atthe wide edge; and γ represents the current zoom relative magnification(magnification information) assuming that the wide edge is one time(×1).

In addition, as shown in FIG. 8B, a three-dimensional rotation matrix isgenerally expressed as follows.

${{Ry}(\phi)} = \begin{bmatrix}{\cos\;\phi} & 0 & {{- \sin}\;\phi} \\0 & 1 & 0 \\{\sin\;\phi} & 0 & {\cos\;\phi}\end{bmatrix}$ ${{Rz}(\theta)} = \begin{bmatrix}{\cos\;\phi} & {{- \sin}\;\phi} & 0 \\{\sin\;\phi} & {\cos\;\phi} & 0 \\0 & 0 & 1\end{bmatrix}$

Since the direction of one point (ξ, η) of a photographed picture thatis panned and tilted by angular information (θ, φ) from the homeposition is the same as the direction of one point (α, β) apart from thehome position, the following relation is satisfied.R _(z)(θ)R _(y)(φ)p=1R _(z)(α)R _(y)(β)e _(x)

When the formula is solved with respect to p, the following relation issatisfied.

$\begin{matrix}{\begin{matrix}{p = {{{IR}_{y}\left( {- \phi} \right)}{R_{x}\left( {\alpha - \theta} \right)}{R_{y}(\beta)}e_{x}}} \\{= {I\begin{bmatrix}{{{\cos\left( {\alpha - \theta} \right)}\cos\;{\phi cos\beta}} + {\sin\;{\phi sin\beta}}} \\{{\sin\left( {\alpha - \theta} \right)}\cos\;\beta} \\{{{- {\cos\left( {\alpha - \theta} \right)}}\sin\;{\phi cos\beta}} + {\cos\;{\phi sin\beta}}}\end{bmatrix}}}\end{matrix}{p = {I\begin{bmatrix}a \\b \\c\end{bmatrix}}}} & (3)\end{matrix}$Thus, ξ and η are obtained as follows.1=1/aξ=−1b/k ₁ =−b/k _(1a)η=1c/k ₂ =c/k _(2a)With the above formula, (ξ, η) projected to the photograph coordinatescan be obtained with coordinate data with an angle (α, β) from the homeposition.ξ=(−sin(α−θ)cos β)/(k ₁(cos(α−θ)cos φ cos β+sin φ sin β))η=(−cos(α−θ)sin φ cos β+cos φ sin β)/(k ₂(cos(α−θ)cos φ cos β+sin φ sinβ))Coordinate data (ξ, η) on the obtained picture by the pan tilter camera3 can be obtained from angular coordinates (α, β) on the virtualspherical surface corresponding to coordinate (s, t) of a panoramapicture. Thus, a panorame picture can be generated.

In contrast, coordinate data with an angle (α, β) can be obtained with(ξ, η) projected to photograph coordinates corresponding to thefollowing formula.Since 1=|p|a=1/√(1+k ₁ ²ξ² +k ₂ ²η²)b=−k ₁ξ/√(1+k ₁ ²ξ² +k ₂ ²η²)c=k ₂η/√(1+k ₁ ²ξ² +k ₂ ²η²)where √( ) represents that the square root of the calculated result in () is obtained.

Form Formula (3), the following relations are satisfied.a=cos(α−θ)−cos φ cos β+sin φ sin βb=sin(α−θ)cos βc=−cos(α−θ)sin φ cos β+cos φ sin βThus, the following relations are satisfied.a sin φ+c sin θ=sin βtan(α−θ)=b/(a cos φ−c sin θ)Thus, the following relations are satisfied.β=sin⁻¹(sin φ/√(1+k ₁ ²ξ² +k ₂ ²η²)+sin θk ₂η/√(1+k ₁ ²ξ² +k ₂ ²η²)α=tan⁻¹(−k ₁ξ/(cos φ−k ₂η sin θ))+θ

Thus, the pan angle α and the tilt angle β can be obtained as follows.(α,β)=(f(ξ,η,θ,φ,γ),g(ξ,η,θ,φ,γ))  (4)

If an error is permitted to some extent, (α, β) can be expressed asfollows.α=θ+(λ/γ)×(ξ/Ny),β=φ+(μ/γ)×(η/Nz)In other words, Eq. (4) can be simplified as follows.(α,β)=(f(ξ,θ,γ),g(η,φ,γ))  (5)

Next, with reference to FIG. 9, a method for calculating angularinformation (α, β) of the pan tilter 28 expressed by Eq. (4) and Eq. (5)with positional coordinates (ξ, η) of the operation area 6A will bedescribed. First of all, an example of a method for directly designatinga desired point in the operation area 6A will be described. Assumingthat the center of the operation area 6A is defined as (0, 0) ofrelative coordinates as shown in FIG. 9A, the positional coordinates (ξ,η) of the mouse pointer of the mouse 8 in the operation area 6A areobtained.

Next, another method for designating a desired point generated with adesired area in the operation area 6A will be described. As shown inFIG. 9A, after a start point (m1, n1) in a desired area is designated,an end point (m2, n2) in the desired area is designated. As thecoordinates at the center of the rectangle generated with these twopoints, a desired point (ξ, η) is obtained as Eq. (6).(ξ,η)=((m1,n1)+(m2,n2))/2  (6)

FIG. 9A shows coordinates of the mouse 8 (pointing device 14) in theoperation area 6A. In FIG. 9A, the moving range (y direction and zdirection) of the mouse pointer of the mouse 8 in the operation area 6Ais denoted by (Ny₁, Nz₁). Angular coordinates (α, β) of the pan tilter28 are obtained with positional coordinates (ξ, η) of the desired point(at the mouse pointer of the mouse 8), angular information (θ, φ) thatrepresents the orientation of the pan tilter 28, and magnificationinformation (γ) of the current zoom relative magnification assuming thatthe wide edge of the zoom lens 16 is defined as one magnificationcorresponding to Eq. (4) or Eq. (5).

The angular coordinates (α, β) shown in FIG. 9B are used to place aposition designated by the pointing device to the center of thephotographed screen assuming that the home position of the pan tilter 28is defined as the origin of latitude and longitude.

The coordinates obtained in FIGS. 9A and 9B may be absolute coordinatesof the screen of the monitor 2 or relative coordinates assuming that thecenter of the operation area 6A is defined as (0, 0). In the coordinatesshown in FIGS. 9A and 9B, coordinates in the pan direction arerepresented by ξ, m1, m2, θ, and α and coordinates in the tilt directionare represented by η, n1, n2, φ, and β.

Thus, when the mouse pointer of the mouse 8 is present in the operationarea 6A, the angular information (α, β) of the pan tilter 28 iscalculated with the angular information (θ, φ) of the current pan tilter28 obtained with received data, the zoom magnification information (γ),and the positional information (ξ, η) at the mouse pointer of the mouse8 corresponding to Eq. (4) or Eq. (5) so that the designated object isplaced at the center of the operation area 6A. The angular coordinates(α, β) of the pan tilter 28 are converted into internal positionalinformation (PNew, TNew) as shown in FIGS. 11A and 11B. The resultantinternal positional information (PNew, TNew) is stored in a send bufferalong with an absolute position drive command of the pan tilter 28. Inaddition, as will be described later, a data send request flag (FlagSo)is set so that data is sent upon occurrence of a timer event.

Next, with reference to FIGS. 10A, 10B, and 10C, a method for convertingpositional coordinates (ξ, η) of the mouse pointer of the mouse 8 in thepanorama operation area 6B of the panorama picture into angularcoordinates (α, β) corresponding to the present invention will bedescribed. As with the method for directly designating a desired pointin the operation area 6A, as shown in FIG. 10A, with a method fordirectly designating a desired point in the panorama operation area 6B,positional coordinates (ξ, η) at the mouse pointer of the mouse 8 can beobtained.

Next, another method for designating a desired point generated with adesired area in the panorama operation area 6B will be described. Asshown in FIG. 10A, after a start point (m1, n1) of a desired area isdesignated, the end point (m2, n2) of the desired area are designated.Corresponding to Eq. (6), a desired point (ξ, η) is obtained.

In FIG. 1A, the moving range (y direction and z direction) of the mousepointer of the mouse 8 in the panorama operation area 6B (the movingrange is defined as the coordinates of the mouse pointer of the mouse 8(pointing device 14) in the panorama operation area 6B) is representedby (Ny₂, Nz₂). The moving range is limited by the pan tilter limiter 6Ddenoted by dotted lines in the panorama operation area 6B. The pantilter limiter 6D represents the moving range of the optical axis of thelens of the pan tilter camera 3. In other words, a point cannot bedesignated out of the pan tilter limiter 6D. Positional coordinates (x,y) in the panorama operation area 6B, angle-of-view information (s, t),and angular information (α, β) of the pan tilter 28 can be obtained withthe positional coordinates (ξ, η) of the desired point, the angularinformation (θ, φ) representing the orientation of the pan tilter 28,and the magnification information (γ) as the current zoom relativemagnification assuming that the wide edge of the zoom lens 16 is definedas one magnification corresponding to Eq. (7), Eq. (8), and Eq. (9).(x,y)=(f ₀(θ),g ₀(f))  (7)(s,t)=(f ₁(γ),g ₁(γ))  (8)(α,β)=(f(ξ),g(η))  (9)

In FIG. 10B, positional coordinates (x, y) represent the currentorientation of the pan tilter 28 assuming that the home position of thepan tilter 28 is defined as the origin of latitude and longitude.Angle-of-view information (s, t) is the current angle of view in theoperation area 6A. FIG. 10B represents the states of the zoom lens andthe pan tilter in the panorama operation area 6B.

In FIG. 10C, angular coordinates (α, β) are used to place the positiondesignated by the pointing device to the center of the photographedscreen assuming that the home position of the pan tilter 28 is definedas the origin of latitude and longitude. (PragMax, TragMax) and(PragMin, TragMin) represent the moving range of the pan tilter (namely,the range represented by the pan tilter limiter 6D). FIG. 10C shows adrive target value in the pan tilter moving range.

In FIGS. 10A, 10B, and 10C, coordinates to be obtained may be absolutecoordinates on the screen of the monitor 2 or relative coordinatesassuming that the center of the panorama operation area 6B is defined as(0, 0). In the coordinates, coordinates in the pan direction arerepresented by ξ, m1, m2, x, s, and α and coordinates in the tiltdirection are represented by η, n1, n2, y, t, and β.

Thus, when the mouse pointer of the mouse 8 is present in the panoramaoperation area 6B, angular information (α, β) of the pan tilter 28 iscalculated with positional information (ξ, η) at the mouse pointer ofthe mouse 8 corresponding to Eq. (9) so that the designated object inthe operation area 6A is placed at the center of the operation area 6A.Angular coordinates (α, β) of the pan tilter 28 are converted intointernal positional information (PNew, TNew) of the pan tilter 28corresponding to the method shown in FIGS. 11A and 11B. The internalpositional information (PNew, TNew) of the pan tilter 28 is stored in asend buffer along with an absolute position drive command of the pantilter 28. In addition, as will be described later, a data send requestflag (FlagSo) is set so that data is sent upon occurrence of the timerevent.

Next, a method for converting internal positional information (p, t) ofthe pan tilter 28 into angular information (α, β) and a method forconverting angular coordinates (α, β) into internal positionalinformation (PNew, TNew) of the pan tilter 28 will be described withreference to FIGS. 11A and 11B. In FIG. 11A, PragMin represents angulardata at the left edge assuming that the home position of the pan tilter28 is 0 (reg). PragMax represents angular data at the right edgeassuming that the home position of the pan tilter 28 is 0 (rag). PdatMinrepresents internal count data at the left edge of the pan tiltercontroller 25. PdatMax represents internal counter data at the rightedge of the pan tilter controller 25.

To obtain the pan angle θ with the pan data p, since the followingrelation is satisfied,(PragMax−θ):(PragMax−PragMin)=(PdatMax−p):(PdatMax−PdatMin)the pan angle θ is expressed as follows.θ=PragMax−(PragMax−PragMin)×(PdatMax−p)/(PdatMax−PdatMin)

Thus, the pan data p is expressed as follows.p=PdatMax−(PragMax−θ)×(PdatMax−PdatMin)/(PragMax−PragMin)

In addition, to obtain the pan data PNew with the pan angle α, since thefollowing relation is satisfied,(PragMax−α):(PragMax−PragMin)=(PdatMax−p-new):(PdatMax−PdatMin)the pan data PNew is expressed as follows.PNew=PragMax−(PragMax−α)×(PdatMax−PdatMin)/(PragMax−PragMin)

In FIG. 11B, TragMin represents angular data at the upper edge assumingthat the home position of the pan tilter 28 is 0 (rag). TragMaxrepresents angular data at the lower edge assuming that the homeposition of the pan tiler 28 is 0 (rag). TdatMin represents internal

count data at the upper edge of the pan tilter controller 25. TdatMaxrepresents internal count data at the lower edge of the pan tiltercontroller 25.

To obtain the tilt angle % with the tilt data t, since the followingrelation is satisfied,(TragMax−φ):(TragMax−TragMin)=(TratMax−t):(TdatMax−TdatMin)the tilt angle φ is expressed as follows.φ=TragMax−(TragMax−TragMin)×(TdatMax−t)/(TdatMax−TdatMin)

Thus, the tilt data t is expressed as follows.t=TdatMax−(TragMax−φ)×(TdatMax−TdatMin)/(TragMax−TragMin)

To obtain the tilt data TNew with the tilt angle β, since the followingrelation is satisfied,(TragMax−β):(TragMax−TragMin)=(TdatMax−t-new):(TdatMax−TdatMin)the tilt data TNew is expressed as follows.TNew=TragMax−(TragMax−β)×(TdatMax−TdatMin)/(TragMax−TragMin)

Next, with reference to FIGS. 12A and 12B, a method for convertingpositional coordinates (ξ, η) in the panorama operation area 6B intoangular coordinates (α, β) of the pan tilter 28 and a method forconverting angular information (θ, φ) of the pan tilter 28 intopositional coordinates (x, y) in the panorama operation area 6B will bedescribed. In FIG. 12A, PragMin represents angular data at the left edgeassuming that the home position of the pan tilter 28 is 0 (rag). PragMaxrepresents angular data at the right edge assuming that the homeposition of the pan tilter 28 is 0 (rag). Ny₂ represents a horizontalcoordinate of the panorama operation area 6B. −Ny₂/2 representscoordinate data at the left edge of the panorama operation area 6B.Ny₂/2 represents coordinate data at the right edge of the panoramaoperation area 6B.

To obtain the pan angle α with the coordinate data ξ, since thefollowing relation is satisfied,(PragMax−α):(PragMax−PragMin)=(Ny ₂/2−ξ):Ny ₂the pan angle α is expressed as follows.α=PragMax−(PragMax−PragMin)×(Ny ₂/2−ξ/Ny ₂

To obtain the coordinate data x with the pan angle θ, since thefollowing relation is satisfied,(PragMax−θ):(PragMax−PragMin)=(Ny ₂/2−x):Ny ₂the coordinate data x is expressed as follows.x=Ny ₂/2−(PragMax−θ)×Ny ₂/(PragMax−PragMin)

In FIG. 12B, TragMin represents angular data at the upper edge assumingthat the home position of the pan tilter 28 is 0 (rag). TragMaxrepresents angular data at the lower edge assuming that the homeposition of the pan tilter 28 is 0 (rag). Nz₂ represents a verticalcoordinate of the panorama operation area 6B. −Nz₂/2 representscoordinate data at the upper edge of the panorama operation area 6B.Nz₂/2 represents coordinate data at the lower edge of the panoramaoperation area 6B.

To obtain the tilt angle β with the coordinate data η, since thefollowing relation is satisfied,(TragMax−β):(TragMax−TragMin)=(Nz ₂/2−η):Nz ₂the tilt angle β is expressed as follows.β=TragMax−(TragMax−TragMin)×(Nz ₂/2−η)/Nz ₂

To obtain the coordinate data y with the tilt angle φ, since thefollowing relation is satisfied,(TragMax−φ):(TragMax−TragMin)=(Nz ₂/2−y):Nz ₂the coordinate data y is expressed as follows.y=Nz ₂/2−(TragMax−θ)×Nz ₂/(TragMax−TragMin)

Next, with reference to FIGS. 13A to 13D, a method for convertingangle-of-view information (ψ, ω) captured by the pan tilter 28 intoangle-of-view information (θ, t) of the frame 6C in the panoramaoperation area 6B will be described. FIG. 13A shows the currentangle-of-view information (ψ, ω) of the pan tilter 28. The angle-of-viewinformation (ψ, ω) is expressed as follows.(ψ,ω)=1/γ×(ψ0,ω0)At this point, (ψ, ω) represent the horizontal angle of view and thevertical angle of view at the wide edge. λ represents the magnificationof the lens assuming that the wide edge is defined as one magnification.

As shown in FIG. 13B, PragMin represents angular data at the left edgeassuming that the home position of the pan tilter 28 is 0 (rag). PragMaxrepresents angular data at the right edge assuming that the homeposition of the pan tilter 28 is 0 (rag). Ny₂ represents a horizontalcoordinate of the panorama operation area 6B. −Ny₂/2 representscoordinate data at the left edge of the panorama operation area 6B.Ny₂/2 represents coordinate data at the right edge of the panoramaoperation area 6B.

To obtain the horizontal angle of view s with the horizontal angle ofview ψ, since the following relation is satisfied,ψ:(PragMax−PragMin)=s:Ny ₂horizontal angle of view s is expressed as follows.s=ψ×Ny ₂/(PragMax−PragMin)

In FIG. 13C, TragMin represents angular data at the lower edge assumingthat the home position of the pan tilter 28 is 0 (rag). TragMaxrepresents angular data at the upper edge assuming that the homeposition of the pan tilter 28 is 0 (rag). Nz₂ represents a verticalcoordinate of the panorama operation area 6B. −Nz₂/2 representscoordinate data at the lower edge of the panorama operation area 6B.Nz₂/2 represents coordinate data at the upper edge of the panoramaoperation area 6B.

To obtain the vertical angle of view t with the vertical angle of viewω, since the following relation is satisfied,ω:(TragMax−TragMin)−t:Nz ₂the vertical angle of view t is expressed as follows.t=ω×Nz ₂/(TragMax−TragMin)

Thus, the angle-of-view information (s, t) shown in FIG. 13D isdisplayed as the frame 6C in the panorama operation area 6B.

Next, with reference to FIG. 14, a method for converting the positionalinformation (z) of the zoom lens 16 into magnification information (γ)will be described. In FIG. 14, the vertical axis represents informationof lens magnification, whereas the horizontal axis represents theinternal information of zoom lens. The positional information (z) of thezoom lens 16 is converted into the magnification information (γ) by thecomputer 1 corresponding to a conversion graph shown in FIG. 14. Forexample, the positional information (z) is converted into themagnification information (γ) corresponding to a ROM table or anequation.

Next, with reference to FIG. 15, an example of a control algorithm ofthe computer 1 will be described. When the program is executed, the flowadvance to step S1. At step S1, the operation area 6, the panoramaoperation area 6B, the cursor 7, and the pan tilter limiter 6D areinitialized and displayed on the monitor 2 as shown in FIG. 2. The rangeof the pan tilter limiter 6D may be fixed or variable. At step S2, atimer is set so that the computer 1 communicates with the modecontroller 23 at predetermined intervals. After such initial setupoperations have been completed, the flow advances to step S3. At stepS3, the system waits for an occurrence of an event. Corresponding to anevent that occurs, the flow advances to a relevant step (for example, atimer event (at step S4), a mouse button down event (at step S5), amouse button up event (at step S6), and a mouse move event (at stepS7)).

Next, with reference to a flow chart shown in FIGS. 16A and 16B, thealgorithm of the timer event will be described. The timer event is anevent for causing the computer 1 to communicate with the mode controller23 at predetermined intervals. The timer event occurs at intervals offor example 50 msec. When the timer event occurs, the flow advances tostep S11. At step S11, the system determines whether or not acommunication port has been set. When the communication port has beenset (namely, the determined result at step S11 is Yes), the flowadvances to step S12. When the communication port has not been set(namely, the determined result at step S11 is No), the flow advances tostep S18. At the first time the communication port has not been set, theflow advances to step S18. At step S18, the system opens thecommunication port. Actually, at step S18, the system opens an RS-232Cport of the computer 1. Thereafter, the flow advances to step S16.

Thereafter, in the timer event, the system performs a receive datachecking process, an analyzing process, a data sending process for datastored in the send buffer (such as the drive command for the pan tilter28), and/or a communication data sending process for state checkrequests for the pan tilter 28 and the zoom lens 16. In this algorithm,the flow advances from step S11 to step S12. At step S12, the systemdetermines whether or not data is stored in the receive buffer. Whendata is stored in the receive buffer (namely, the determined result atstep S12 is Yes), the flow advances to step 313. When data is not storedin the receive buffer (namely, the determined result at step S12 is No),the flow advances to step S14. At step S13, the system analyzes receivedata stored in the receive buffer and obtains positional information (p,t) of the pan tilter 28 and positional information (z) of the zoom lens16 that have been requested to the mode controller 23. The systemconverts the positional information (p, t) of the pan tilter 28 and thepositional information (z) of the zoom lens 16 into angular information(θ, φ) of the pan tilter 28 and magnification information (γ) of thezoom lens 16 corresponding to methods shown in FIGS. 11 and 14.

At step S14, the system determines whether or not a data send requesthas been issued. When a data send request has been issued (FlagSo==True)(namely, the determined result at step S14 is Yes), the flow advances tostep S19. At step S19, the system sends data stored in the send bufferand resets the send request flag (FlagSo==False). Next, the flowadvances to step S16. An example of data stored in the send buffer isdata of a drive command of the pan tilter 28 designated with the mouse8. When a send request has not been issued (FlagSo==False) (namely, thedetermined result at step S14 is No), the flow advances to step S15. Atstep S15, the system sends position request commands for the pan tilter28 and the zoom lens 16 from the computer 1 to the mode controller 23.

At step S16, the system compares the old positional information of thepan tiler 28 with the new positional information thereof and determineswhether or not the positional information (p, t) has varied. When thepositional information (p, t) of the pan tilter 28 has varied (namely,the determined result at step S16 is Yes), the flow advances to stepS20. When the positional information (p, t) of the pan tilter 28 has notvaried (namely, the determined result at step S16 is No), the flowadvances to step S17. At step S17, the system compares the oldpositional information of the zoom lens 16 with the new positionalinformation thereof and determines whether or not the positionalinformation (z) has varied. When the positional information (z) of thezoom lens 16 has varied (namely, the determined result at step S17 isYes), the flow advances to step S20. When the positional information (z)of the zoom lens 16 has not varied (namely, the determined result atstep S17 is No), this event is completed.

At step S20, when the positional information (p, t) of the pan tilter 28and/or the positional information (z) of the zoom lens 16 has varied,the system redraws the frame 6C in the panorama operation area 6B. Atthis point, the system converts the positional information (p, t) of thepan tilter 28 into the angular information (θ, φ). In addition, thesystem converts the positional information (z) of the zoom lens 16 intothe magnification information (γ). With the converted angularinformation (θ, φ) and magnification information (γ), the systemcalculates positional coordinates (x, y) of the pan tilter 28 andangle-of-view information (s, t) that is the angle of view displayed inthe operation area 6A corresponding to Eq. (7) and Eq. (8),respectively. Corresponding to the resultant positional coordinates (x,y) and angle-of-view information (s, t), the system draws the frame 6Cin the panorama operation area 6B.

At step S16, the system compares the old positional information (p, t)of the pan tilter 28 with the new positional information (p, t) thereof.Alternatively, the system may compare the old angular information (θ, φ)of the pan tilter 28 with the new angular information (6, 1) thereof. Inthis case, at step S20, with the new angular information (θ, φ), thesystem calculates the positional coordinates (x, y) corresponding to Eq.(7). Likewise, at step S17, the system compares the old positionalinformation (z) of the zoom lens 16 with the new positional information(z) thereof. Alternatively, the system may compare the old magnificationinformation (γ) of the zoom lens 16 with the new magnificationinformation (γ) thereof. In this case, at step S20, the systemcalculates the angular information (s, t) with the new magnificationinformation (γ) corresponding to Eq. (8).

Next, with reference to a flow chart shown in FIG. 17, the algorithm ofthe mouse move event will be described. The mouse move event is an eventthat occurs when the mouse 8 is moved. According to the presentinvention, the mouse move event is used to select a drive position ofthe pan tilter 28. When the mouse move event occurs, the flow advancesto step S21. At step S21, the system determines whether or not the mousepointer of the mouse 8 is present in the operation area 6A, the panoramaoperation area 6B, or the other area. When the mouse pointer of themouse 8 is present in the operation area 6A (namely, the determinedresult at step S21 is Yes), the flow advances to step S22. When themouse pointer of the mouse 8 is not present in the operation area 6A(namely, the determined result at step S21 is No), the flow advances tostep S24. At step S22, the system sets an operation area flag(Flag-rin==True) and clears a panorama operation area flag(Flag-pin==False).

At step S24, since the mouse pointer of the mouse 8 is not present inthe operation area 6A, the system clears the operation area flag(Flag-rin==False). At step S25, the system determines whether or not themouse pointer of the mouse 8 is present in the panorama operation area6B. When the mouse pointer of the mouse 8 is present in the panoramaoperation area 6B (namely, the determined result at step S25 is Yes) theflow advances to step S26. When the mouse pointer of the mouse 8 is notpresent in the panorama operation area 6B (namely, the determined resultat step S25 is No), the flow advances to step S27. At step S26, thesystem sets the panorama operation area flag (Flag-pin==True). At stepS27, since the mouse pointer of the mouse 8 is not present in thepanorama operation area 6B, the system clear the panorama operation areaflag (Flag-pin==False).

When the mouse pointer of the mouse 8 is present in the operation area6A or the panorama operation area 6B (namely, the determined result atstep S21 or step S25 is Yes), at step S23, the system obtains positionalcoordinates (ξ, η) of the mouse pointer of the mouse 8 assuming that thecenter of the operation area is defined as (0, 0) of relativecoordinates.

In this flow chart, when the mouse pointer of the mouse 8 is present inthe operation area 6A (namely, the determined result at step S22 isYes), the system sets the operation area flag (Flag-rin==True).

When the mouse pointer of the mouse 8 is not present in the operationarea 6A (namely, the determined result at step S22 is No), the systemclears the operation area flag (Flag-rin==False). When the mouse pointerof the mouse 8 is present in the panorama operation area 6B (namely, thedetermined result at step S25 is Yes), the system sets the panoramaoperation area flag (Flag-pin==True). When the mouse pointer 8 is notpresent in the panorama operation area 6A (namely, the determined resultat step S25 is No), the system clears the panorama operation area flag(Flag-pin==False). When the mouse pointer of the mouse 8 is present inthe operation area 6A or the panorama operation area 6B (namely, thedetermined result at step S21 or S35 is Yes), the system designates thepositional coordinates of the mouse pointer of the mouse 8 to (ξ, η)assuming that the center of each operation area is defined as (0, 0) ofrelative coordinates.

Next, the mouse button down event and the button up event will bedescribed. In the method for directly designating a desired point of theoperation area 6A or the panorama operation area 6B, only the algorithmof a mouse button down event shown in FIG. 18 is used. In the method fordesignating a desired point generated with a desired area, both thealgorithm of a mouse button down event shown in FIG. 19 and thealgorithm of a mouse button up event shown in FIG. 20 are used.

With reference to a flow chart shown in FIG. 18, the algorithm of thebutton down event for the method for directly designating a desiredpoint of the operation area will be described. This event is an eventthat occurs when the left button of the mouse 8 is pressed. In thepresent invention, this event is used as trigger information for drivingthe pan tilter 28. When this event occurs, the flow advances to stepS31. At step S31, the system determines whether or not the mouse pointer8 is present in the operation area 6A corresponding to the operationarea flag. When the operation area flag has been set (FlagRin==True)(namely, the determined result at step S31 is Yes), since the mousepointer of the mouse 8 is present in the operation area 6A, the flowadvances to step S32.

When the operation area flag has been cleared (FlagRin==False) (namely,the determined result at step S31 is No), since the mouse pointer of themouse 8 is not present in the operation area 6A, the flow advances tostep S34.

When the mouse pointer of the mouse 8 is present in the operation area6A (namely, the determined result at step S31 is Yes), the flow advancesto step S32. At step S32, the system calculates angular information (α,β) of the pan tilter 28 with the angular information (θ, φ) of thecurrent pan tilter 28 obtained from the received data, the magnificationinformation (γ) of the zoom lens 16; and the positional coordinate (θ,η) of the mouse pointer of the mouse 8 in the operation area 6Acorresponding to Eq. (4) or Eq. (5) so that the designated object in theoperation area is placed at the center of the screen.

At step S33, the system converts the angular information (α, α) of thepan tilter 28 into the internal positional information (PNew, TNew)corresponding to the method shown in FIG. 11. The system stores theconverted positional information (PNew, TNew) in the send buffer alongwith the absolute position drive command of the pan tilter 28. Inaddition, the system sets the data send request flag (FlagSo==True) andsends the data with the process of the timer event.

After the system has determined that the mouse pointer of the mouse 8 isnot present in the operation area 6A (namely, the determined result atstep S31 is No), the flow advances to step S34. At step S34, the systemdetermines whether or not the mouse pointer of the mouse 8 is present inthe panorama operation area 6B corresponding to the panorama operationarea flag. When the panorama operation flag has been set (FlagPin==True)(namely, the determined result at step S34 is Yes), since the mousepointer of the mouse 8 of the panorama operation area 6B is present inthe panorama operation area 6B, the flow advances to step S35. When thepanorama operation flag has been cleared (FlagPin==False) (namely, thedetermined result at step S34 is No), this event is completed.

In this flow chart, the system determines whether or not the mousepointer of the mouse 8 is present in the operation area 6A or thepanorama operation area 6B corresponding to the operation area flag(FlagRin) and the panorama operation area flag (FlagPin). When the mousepointer of the mouse 8 is not present in the operation area 6A and thepanorama operation area 6B, this event becomes invalid.

When the mouse pointer of the mouse 8 is present in the panoramaoperation area 6B (namely, the determined result at step S34 is Yes),the flow advances to step S35. At step S35, the system calculatesangular information (α, β) of the pan tilter 28 with the positionalinformation (ξ, η) at the mouse pointer of the mouse 8 in the panoramaoperation area 6B corresponding to Eq. (9) so that the designated objectin the operation area is placed at the center of the screen. Thereafter,the flow advances to step S33.

Next, with reference to FIGS. 19 and 20, the algorithms of the buttondown event and the button up event for the method for designating adesired point generated with a desired area in the panorama operationarea 6B will be described, respectively.

With reference to the flow chart shown in FIG. 19, the algorithm of thebutton down event will be described. This event is an event that occurswhen the left button of the mouse 8 is pressed. In this embodiment, thisevent is used as an event for determining the start point of a desiredarea. When this event occurs, the flow advances to step S41. At stepS41, the system determines whether or not the mouse pointer of the mouse8 is present in the operation area 6A corresponding to the operationarea flag (FlagRin). When the operation area flag has been set(FlagRin==True) (namely, the determined result at step S41 is Yes),since the mouse pointer of the mouse 8 is present in the operation area6A, the flow advances to step S42. When the operation area flag has beencleared (FlagRin==False) (namely, the determined result at step S41 isNo), since the mouse pointer of the mouse 8 is not present in theoperation area 6A, the flow advances to step S44.

When the mouse pointer of the mouse 8 is present in the operation area6A (namely, the determined result at step S41 is Yes), at step S42, thesystem sets an operation area start point obtain flag (FlagRstart=True).Thereafter, the flow advances to step S43. At step S43, the systemstores positional coordinates (m1, n1) at which the left button of themouse 8 is pressed as the start point of the desired area.

After the system has determined that the mouse pointer of the mouse 8 isnot present in the operation area 6A, at step S44, the system determineswhether or not the mouse pointer of the mouse 8 is present in thepanorama operation area 6B corresponding to the panorama operation areaflag (FlagPin). When the panorama operation area flag has been set(FlagPin==True) (namely, the determined result at step S44 is Yes),since the mouse pointer of the mouse 8 is present in the panoramaoperation area 6B, the flow advances to step S45. When the panoramaoperation area flag has been cleared (FlagPin==False) (namely, thedetermined result at step S44 is No), this event is completed.

In this flow chart, the system determines whether or not the mousepointer of the mouse 8 is present in the operation area 6A or thepanorama operation area 6B corresponding to the operation area flag(FlagRin) and the panorama operation area flag (FlagPin). When the mousepointer of the mouse 8 is not in the operation area 6A and the panoramaoperation area 6B, this event becomes invalid.

When the mouse pointer of the mouse 8 is present in the panoramaoperation area 6B (namely, the determined result at step 44 is Yes), theflow advances to step S45. At step S45, the system sets a panoramaoperation area start point obtain flag (FlagPstart). Thereafter, theflow advances to step S43.

Next, with reference to a flow chart shown in FIG. 20, the algorithm ofthe button up event will be described. This event is an event thatoccurs when the left button of the mouse 8 is released. In the presentinvention, the button up event is used as an event for determining theend point of a desired area.

When this event occurs, the flow advances to step S51. At step S51, thesystem determines whether or not the operation area flag has been set(FlagRin==True) (namely, the mouse pointer of the mouse 8 is present inthe operation area 6A). When the operation area flag has been set(FlagRin=True) (namely, the determined result at step S51 is Yes), sincethe mouse pointer of the mouse 8 is present in the operation area 6A,the flow advances to step S52. When the operation area flag has beencleared (FlagRin==False) (namely, the determined result at step S51 isNo), since the mouse pointer of the mouse 8 is not present in theoperation area 6A, the flow advances to step S57. At step S52, thesystem determines whether or not the left button of the mouse 8 has beenpressed in the operation area 6A corresponding to an operation areastart point obtain flag (FlagRstart). When the start point obtain flaghas been set (FlagRstart==True) (namely, the determined result at stepS52 is Yes), since the left button of the mouse 8 has been pressed inthe operation area 6A, the flow advances to step S53. When the startpoint obtain flag has been cleared (FlagRstart==False) (namely, thedetermined result at step S52 is No), since the left button of the mouse8 has not been pressed in the operation area 6A, the flow advances tostep S57.

In other words, at steps S51 and S52, the system determines whether ornot the operation area flag and the operation area start point obtainflag have been set or cleared. When the operation area flag and thestart point obtain flag have been set (FlagRin==True andFlagRstart==True), the system determines that the drive command hastaken place in the operation area 6A. Otherwise, at steps S57 and S58,the system determines whether or not the panorama operation area flag(FlagPin) and the panorama operation area start point obtain flag(FlagPstart) have been set or cleared.

When the drive command has taken place in the operation area (namely,the operation area flag and the start point obtain flag have been set(FlagRin==True and FlagRstart==True), at step S53, the system stores thepositional coordinates (m2, n2) at which the left button of the mouse 8has been released in the operation area 6A as the end point of thedesired area. Thereafter, the system calculates positional information(ξ, η) as the coordinates of the center of the rectangle area generatedwith the positional coordinates (m1, n1) of the start point of thedesired area and the positional coordinates (m2, n2) of the end pointthereof.

At step S54, the system calculates angular information (α, β) of the pantilter 28 with the angular information (θ, φ) of the pan tilter obtainedfrom the received data, the magnification information (γ) of the zoomlens 16, and the positional information (ξ, η) at the mouse pointer ofthe mouse 8 corresponding to Eq. (4) or Eq. (5).

At step S55, the system converts the angular information (α, β) of thepan tilter 28 into the internal positional information (PNew, TNew) ofthe pan tiler 28 corresponding to the method shown in FIG. 11 and storesthe positional information (PNew, TNew) to the send buffer along withthe absolute position drive command. In addition, the system sets thedata send request flag (FlagSo==True) and sends data with the process ofthe timer event.

At step S56, after the system has checked the mouse button up event ineach operation area, the system clears the operation area start pointobtain flag and the panorama operation area start point obtain flag(FlagRstart==False and FlagPstart==False). Thereafter, this event iscompleted.

At step S57, the system determines whether or not the mouse pointer ofthe mouse 8 is present in the panorama operation area 6B correspondingto the panorama operation area flag (FlagPin). When the panoramaoperation area flag has been set (FlagPin==True) (namely, the determinedresult at step S57 is Yes), since the mouse pointer of the mouse 8 ispresent in the panorama operation area 6B, the flow advances to stepS58. When the panorama operation area flag has not been set(FlagPin==False), since the mouse pointer of the mouse 8 is not presentin the panorama operation area 6B, the flow advances to step S56. Atstep S58, the system determines whether or not the left button of themouse 8 has been pressed in the panorama operation area 6B correspondingto the panorama operation area start point obtain flag (FlagPstart).When the start point obtain flag has been set (FlagPstart==True)(namely, the determined result at step S58 is Yes), since the leftbutton of the mouse 8 has been pressed in the panorama operation area 6,the flow advances to step S59. When the start point obtain flag has notbeen set (FlagPstart==False) (namely, the determined result at step S58is No), since the left button of the mouse 8 has not been pressed in thepanorama operation area 6B, the flow advances to step S56.

When the panorama operation area flag and the panorama operation startpoint obtain flag have been set (FlagPin==True and FlagPstart==True) atsteps S57 and S58, the system determines that a drive command has issuedin the panorama operation area 6B. When the conditions at steps S51,S52, and S58 are not satisfied, this event becomes invalid.

When the drive command has been issued in the panorama operation area 6B(namely, the panorama operation area flag and the start obtain flag havebeen set (FlagPin==True and Flag-pstart==True), the flow advances tostep S59. At step S59, the system stores the positional coordinates (m2,n2) at which the left button of the mouse 8 has been released in thepanorama operation area 6B as the end point of the desired area. Thesystem calculates the positional information (ξ, η) of the mouse pointerof the mouse 8 as the coordinates of the center of the rectangle areawith the positional coordinates (m1, n1) of the start point of thedesired area that has been stored and the positional coordinates (m2,n2) of the end point of the desired area corresponding to Eq. (6).

At step S60, the system calculates angular information (α, β) of the pantilter 28 with the positional information (ξ, η) at the mouse pointer ofthe mouse 8 in the panorama operation area 6B corresponding to Eq. (9)so that the designated object in the panorama operation area is placedat the center of the screen. Thereafter, the flow advances to step S55.

In the above-described embodiment, one computer performs all processesof the system. On the other hand, according to a second embodiment ofthe present invention, as shown in FIG. 21, processes are shared by aserver computer and a client computer so as to control a pan tilercamera through a network that has a restriction of a communicationcapacity. In FIG. 21, a computer 1 is connected to a monitor 2 and amouse 8. The computer 1 controls the operation of a pan tilter camera 3disposed at a remote place through a transmission line and a server 9.In other words, the computer 1 composes a controller for a photographingapparatus. The transmission line may be a communication line (radiocommunication line or a cable communication line), a network, or thelike. The computer 1 has a relation of a client to the server 9. Aplurality of computers 1 can be connected to the server 9.

The pan tilter camera 3 and the server 9 are disposed on a real scene inan environment denoted by reference numeral 4. A screen photographed bythe pan tilter camera 3 disposed on the real scene 4 is denoted byreference numeral 5. Hereinafter, the screen 5 is referred to asphotographed screen. The photographed screen 5 is an actuallyphotographed screen. When the zoom lens is placed on the telephotographside, the angle of view decreases. In contrast, when the zoom lens isplaced on the wide-angle side, the angle of view increases.

A picture photographed by the pan tilter camera 5 is sent to a server 9.The server 9 converts the photographed picture into video data. Thevideo data is sent to the computer 1 through a transmission line. Thevideo data sent to the computer 1 is decoded and displayed on themonitor 2. The monitor 2 displays the photographed screen 5 in theoperation area 6A thereof. A panorama picture with which a picturephotographed by the pan tilter camera 3 is superimposed is displayed inthe panorama operation area 6B. As with the above-described embodiment,a desired point of the panorama operation area 6B (or the operation area6A) or a desired point generated with a desired point is designated withthe mouse 8 (cursor 7). The pan tilter camera 3 is driven through theserver 9 and the transmission line and thereby the photographed screenis moved. In other words, the pan tilter camera 3 is controlled throughthe server 9 so that the selected object is placed at the center of theoperation area 6A.

FIG. 22 is a block diagram showing the overall system of the secondembodiment of the present invention. Since the structures and functionsof the camera portion 11 and the pan tilter portion 12 are the same asthose of the first embodiment, the structures thereof are omitted inFIG. 22. The server 9 comprises a controlling portion 131, a videocapture portion 129, and a storing portion 130. The video captureportion 129 is composed of a video capture board. The computer 1 isconnected to a transmission path 132 through a network. The computer 1is composed of a controlling portion 31 and so forth as with the firstembodiment. Since the algorithms used in the computer 1 are the same asthose of the first embodiment, for simplicity, their description isomitted.

Rays emitted from an object are sent to the camera portion 11 as withthe first embodiment. The camera portion 11 converts the rays intovarious signals such as a brightness signal (Y), a color signal (C), anda video signal and supplies the resultant signals as picture signals toa TV monitor 13 and the video capture portion 129 of the server 9. Aswith the first embodiment, the pan tilter portion 12 has a modecontroller, a camera controller, and a pan tilter controller. Thesecontrollers control the camera portion 11 and the pan tilter 28. Themode controller 23 controls the overall system corresponding to theinternal states of the camera portion 11 and the pan tilter portion 12and an external command as with the first embodiment.

The mode controller 23 is connected to the server 9 through acommunication path (in reality, RS232C interface). The mode controller23 sends commands received from the server 9 and commands received fromthe computer 1 through the server 9 to the pan tilter controller and thecamera controller so as to drive the pan tilter and the zoom lens of thelens block portion. The mode controller 23 always receives informationfrom the pan tilter controller and the camera controller so as to sendthe inner state of the pan tilter camera to the outside through theserver 9.

The server 9 obtains the inner state of the pan tilter camera (forexample, the current positional information of the pan tilter and thezoom lens, and so forth) from the mode controller 23 of the pan tilterportion 12 at predetermined intervals. To send a picture photographed bythe camera portion 11 to the transmission path 132, the video captureportion 129 is used. The video capture portion 129 converts a picturesignal received from the camera portion 11 to digital picture data thatis sent to the transmission path 132 in any quality (in the presentembodiment, still picture JPEG format or still picture bit map format).The resultant digital picture is stored in a storing portion 130 (forexample, a hard disk).

When the computer 1 issues a connection request to the server 9, theserver 9 sends a GUI (Graphical User Interface) panel information to thecomputer 1 so as to display a picture on the monitor 2. The panelinformation is an arrangement of a panel and a program that runs on thecomputer 1 when the mouse is operated on the panel. Examples of thepanel information are programs written in HTML, JAVA, and so forth.Picture data photographed by the pan tilter camera and the state thereofare sent to the computer 1 through the transmission path 132 atpredetermined intervals.

In another embodiment, Internet is used as the transmission path 132.Data is exchanged on the transmission path 132 using the HTTP protocol.The computer 1 causes the monitor 2 to display GUI panel information,picture information, the state of the pan information, pictureinformation, the state of the pan tilter camera, and so forth receivedfrom the server 9 with an Internet browser. An operation area 6A, apanorama operation area 6B, a panorama picture generation button 6E,zoom operation buttons, a cursor 7 of a pointing device 14 (mouse 8),and so forth displayed on the GUI panel of the monitor 2. Picture datareceived from the server is decoded and displayed in the operation area6A. When the picture data is updated, the picture is also rewritten inthe operation area 6A. The moving range of the pan tilter camera, theposition of the pan tilter, angle-of-view of the zoom, and so forth aredisplayed on the panorama operation area 6B, with the same method as thefirst embodiment. The computer 1 executes the operation program for theGUI panel received from the server 9.

In the second embodiment of the present invention, a drive command ofthe pan tilter camera 3 and an operation command of the server 9 aregenerated with a clicking operation of the mouse 8. When the mouse 8 isclicked on the panorama generation button 6E, the computer 1 causes theserver 9 to generate a panorama screen. When the server 1 receives thiscommand, as with the first embodiment, the server 9 moves the pan tilterand the zoom lens to relevant positions, photographs ten pictures atthese positions, maps them to the virtual spherical surface, normalizesthem with latitude and longitude, and combines them. After the server 9has combined these pictures as a panorama picture, it converts thepanorama picture into a JPEG format picture. The servers 9 sends theresultant picture to the computer 1 through the transmission line 132.

The computer 1 displays the received panorama picture in the panoramaoperation area 6B of the monitor 2. Thus, the user can see theenvironment at the position of the pan tiler camera 3 at a glace. Whenthe mouse 8 is clicked in the panorama operation area 6B, the computer 1sends to the server 9 a command (absolute position drive command) thatcauses the position at which the mouse is clicked on the panoramapicture to be placed at the center of the operation area 6A (picture).The server 9 sends this command to the pan tilter camera 3. Thus, thepan tilter is driven to a relevant position. In such a manner, the drivetarget of the pan tilter is designated on the panorama screen.Consequently, the user can easily operate the pan tilter without need toconsider a drive command on the network, a delay of a video signal, andso forth.

In the first embodiment, whenever the pan tilter camera 3 sends apicture to the computer 1, the computer 1 combines it and displays thecombined picture in the panorama operation area 6B. Alternatively, afterthe computer has combined all pictures, it may display the resultantpicture in the panorama operation area 6B.

According to the first embodiment, the operation area 6A and thepanorama operation area 6B are displayed on the monitor 2 connected tothe computer 1. Alternatively, the operation area 6A and/or the panoramaoperation area 6B may be displayed on another display unit other thanthe monitor 2.

According to the first embodiment, the pan tilter camera 3 is driven byoperating the operation area 6A and the panorama operation area 6B withthe mouse 8. Alternatively, one of the operation area 6A and thepanorama operation area 6B may be operated with the mouse 8.

According to the first embodiment, the operation area 6A and thepanorama operation area 6B are displayed on the monitor 2.Alternatively, only the panorama operation area 6B may be displayed onthe monitor 2.

According to the first embodiment, the operation area 6A and thepanorama operation area 6B are displayed on the monitor 2. By operatingthe operation area 6A and the panorama operation area 6B with the mouse8, the pan tilter camera 3 is freely driven. Alternatively, a panoramapicture may be displayed on the monitor 2. In this case, the pan tiltercamera 3 may be driven with an operation portion such as eight-directionkeys.

According to the above-described embodiments, the photographing range ofthe pan tilter camera 3 may be the maximum moving range of the pantilter camera 3 or limited with a limiter. The function for limited thephotographing range with the limiter may be provided by the pan tiltercamera 3 or the computer 1.

In the first embodiments, a desired point generated with a desired areais placed at the center of thereof. Alternatively, a desired point maybe placed at for example the center of gravity, the incenter, thecircumcenter, or the orthocenter of the area.

According to the first embodiment, a panorama picture displayed in thepanorama operation area 6B is not limited as long as it represents theenvironment in which the pan tilter camera 3 is disposed. For example,the panorama picture may be a moving picture, an intermittent stillpicture, or a still picture.

According to the second embodiment, for simplicity, one computer 1 isconnected to the remote server 9 and the pan tilter camera 3 that aredisposed at a remote place. Alternatively, a plurality of servers 9 anda plurality of pan tilter cameras 3 may be disposed worldwide. Forexample, one pan tilter camera 3 may be controlled by a plurality ofcomputers through for example Internet.

According to the present invention, with a panorama picture, the usercan see the environment in which the photographing apparatus is disposedat a glance. Since the positional information of the pan tilter, theangle of view of the zoom lens, and the moving range of the pan tilterare added as information to the picture, the user can easily know thestate of the photographing apparatus.

In addition, when the user designates a desired object in the panoramaoperation area, he or she can easily capture it in the field of view ofthe picture to be photographed. Moreover, by designating an object inthe operation area, the user can precisely adjust the position thatcannot be designated in the panorama operation area. In comparison withthe conventional method of which the user operates direction keys whileobserving a monitored picture (namely, a picture is photographed througha feed-back operation and an experience), according to the presentinvention, a desired object can be displayed at the center of theoperation area with the clicking operation of the mouse.

In addition, according to the present invention, since the position towhich the pan tilter moves can be predicted beforehand, on acommunication line that causes picture and information data to bedelayed and/or lost (such as Internet), the user can seamlessly operatethe pan tilter camera. Thus, according to the present invention, the pantilter camera can be easily operated with high visibility.

Although the present invention has been shown and described with respectto a best mode embodiment thereof, it should be understood by thoseskilled in the art that the foregoing and various other changes,omissions, and additions in the form and detail thereof may be madetherein without departing from the spirit and scope of the presentinvention.

What is claimed is:
 1. A signal processing apparatus, comprising: adisplay control unit controlling a display unit to display a first imageand a second image on one screen, the first image being based on imagedata generated by an imaging device; and a position detection unitdetecting a selected position on the first image; wherein the secondimage corresponds to the selected position; wherein the second image isdisplayed so that the second image is larger than a corresponding imagesection in the first image; wherein the corresponding image section is asection within the first image corresponding to the second image.
 2. Thesignal processing apparatus according to claim 1, wherein: a mark isdisplayed on the first image and the mark indicates a position of thecorresponding image section.
 3. The signal processing apparatusaccording to claim 1, wherein: the display unit displays the secondimage so that an object corresponding to a designated point is shown atsubstantially a center of the second image.
 4. The signal processingapparatus according to claim 1, wherein the signal processing apparatusis to be used in a surveillance system.
 5. The signal processingapparatus according to claim 1, wherein the imaging device is connectedto a network and the generated image data is transmitted via thenetwork.
 6. A signal processing system comprising: an imaging device; adisplay control unit controlling a display unit to display a first imageand a second image on one screen, the first image being based on imagedata generated by an imaging device; and a position detection unitdetecting a selected position on the first image; wherein the secondimage corresponds to the selected position; wherein the second image isdisplayed so that the second image is larger than a corresponding imagesection in the first image; wherein the corresponding image section is asection within the first image corresponding to the second image.
 7. Thesignal processing system according to claim 6 wherein: a mark isdisplayed on the first image and the mark indicates a position of thecorresponding image section.
 8. The signal processing system accordingto claim 6, further comprising: a display device.
 9. The signalprocessing system according to claim 6, wherein: the display unitdisplays the second image so that an object corresponding to adesignated point is shown at substantially a center of the second image.10. The signal processing system according to claim 6, wherein: thesignal processing system is a surveillance system.
 11. The signalprocessing system according to claim 6, wherein: the imaging device isconnected to a network and the generated image data is transmitted viathe network.
 12. A signal processing apparatus, comprising: displaycontrol means for controlling a display unit to display a first imageand a second image on one screen, the first image being based on imagedata generated by an imaging device; and position detection means fordetecting a selected position on the first image; wherein the secondimage corresponds to the selected position; wherein the second image isdisplayed so that the second image is larger than a corresponding imagesection in the first image; wherein the corresponding image section is asection within the first image corresponding to the second image. 13.The signal processing apparatus according to claim 12, wherein: a markis displayed on the first image and the mark indicates a position of thecorresponding image section.
 14. The signal processing apparatusaccording to claim 12, wherein: the display unit displays the secondimage so that an object corresponding to a designated point is shown atsubstantially a center of the second image.
 15. The signal processingapparatus according to claim 12, wherein the signal processing apparatusis to be used in a surveillance system.
 16. The signal processingapparatus according to claim 12, wherein the imaging device is connectedto a network and the generated image data is transmitted via thenetwork.
 17. A signal processing method, comprising: controlling adisplay unit to display a first image and a second image on one screen,the first image being based on image data generated by an imagingdevice; and detecting a selected position on the first image; whereinthe second image corresponds to the selected position; wherein thesecond image is displayed so that the second image is larger than acorresponding image section in the first image; wherein thecorresponding image section is a section within the first imagecorresponding to the second image.
 18. The signal processing methodaccording to claim 17, wherein: a mark is displayed on the first imageand the mark indicates a position of the corresponding image section.19. The signal processing method according to claim 17, wherein: thesecond image is displayed so that an object corresponding to adesignated point is shown at substantially a center of the second image.20. The signal processing method according to claim 17, wherein thesignal processing method is carried out in a surveillance system. 21.The signal processing method according to claim 17, wherein the imagingdevice is connected to a network and further comprising the step oftransmitting the generated image data via the network.
 22. Anon-transitory computer readable storage medium on which is recorded aprogram that, when executed by a computer causes the computer to performthe signal processing method, comprising: controlling a display unit todisplay a first image and a second image on one screen, the first imagebeing based on image data generated by an imaging device; and detectinga selected position on the first image; wherein the second imagecorresponds to the selected position; wherein the second image isdisplayed so that the second image is larger than a corresponding imagesection in the first image; wherein the corresponding image section is asection within the first image corresponding to the second image. 23.The medium according to claim 22, wherein; a mark is displayed on thefirst image and the mark indicates a position of the corresponding imagesection.
 24. The medium according to claim 22, wherein; the second imageis displayed so that an object corresponding to a designated point isshown at substantially a center of the second image.
 25. The mediumaccording to claim 22, wherein the signal processing method is carriedout in a surveillance system.
 26. The medium according to claim 22,wherein the imaging device is connected to a network and furthercomprising the step of transmitting the generated image data via thenetwork.
 27. The signal processing apparatus according to claim 5,further comprising: output unit outputting a control data for theimaging device, wherein the control data is transmitted via a network.28. The signal processing apparatus according to claim 27, wherein theimaging device generates an image data based on the control data. 29.The signal processing apparatus according to claim 27, wherein thedisplay unit displays the second image so that an object correspondingto a designated point is shown at substantially a center of the secondimage.
 30. The signal processing system according to claim 6, furthercomprising: output unit outputting a control data for the imagingdevice, wherein the control data is transmitted via a network.
 31. Thesignal processing system according to claim 30, wherein the imagingdevice generates an image data based on the control data.
 32. The signalprocessing system according to claim 30, wherein the display unitdisplays the second image so that an object corresponding to adesignated point is shown at substantially a center of the second image.33. The signal processing apparatus according to claim 12, furthercomprising: output unit outputting a control data for the imagingdevice, wherein the control data is transmitted via a network.
 34. Thesignal processing apparatus according to claim 33, wherein the imagingdevice generates an image data based on the control data.
 35. The signalprocessing apparatus according to claim 33, wherein the display unitdisplays the second image so that an object corresponding to adesignated point is shown at substantially a center of the second image.36. The signal processing method according to claim 17, furthercomprising the steps of outputting a control data for the imagingdevice, wherein the control data is transmitted via a network.
 37. Thesignal processing method according to claim 36, wherein the imagingdevice generates an image data based on the control data.
 38. The signalprocessing method according to claim 36, further comprising the steps ofdisplaying the second image so that an object corresponding to adesignated point is shown at substantially a center of the second image.39. The medium according to claim 22, further comprising: outputting acontrol data for the imaging device, wherein the control data istransmitted via a network.
 40. The medium according to claim 39, whereinthe imaging device generates an image data based on the control data.41. The medium according to claim 39, further comprising the step ofdisplaying the second image so that an object corresponding to adesignated point is shown at substantially a center of the second image.